Papers with recovery phase
RankAdaptor: Hierarchical Rank Allocation for Efficient Fine-Tuning Pruned LLMs via Performance Model (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current compression techniques entail structural pruning and a recovery phase that leverages the Low-Rank Adaptation algorithm. |
| Approach: | They propose a hierarchical rank allocation method that enables efficient fine-tuning of pruned LLMs according to layerwise specific recovery requirements. |
| Outcome: | The proposed algorithm outperforms state-of-the-art methods across pruning settings and LLM architectures with improvements ranging from 0.7% to 5.5%. |